feat: add DSV4 B300 Dynamo-SGLang MTP configuration / 新增 DSV4 B300 Dynamo-SGLang MTP 配置 - #2363
Conversation
中文：新增 DSV4 FP4 B300 Dynamo-SGLang MTP 配置，并接入启用聊天模板输入和 EAGLE 投机解码的共享 srt-slurm 配方矩阵。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
中文：将性能变更日志链接更新为 #2363。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| elif [[ $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX == "dsv4" && $PRECISION == "fp4" ]]; then | ||
| git clone --branch main --single-branch https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR" | ||
| cd "$SRT_REPO_DIR" || exit 1 |
There was a problem hiding this comment.
🟡 The new dynamo-sglang + dsv4 + fp4 branch clones srt-slurm with git clone --branch main --single-branch ... but never pins a commit afterward, unlike the closest analogous case (minimaxm3 mtp) which also clones main but then checks out a fixed SHA. Since this benchmark's CONFIG_FILE entries depend on the recipes/dsv4-pro/sglang/b300-fp4/8k1k-mtp.yaml matrix living on upstream main, a future unrelated commit to NVIDIA/srt-slurm could silently change or break this recipe with no corresponding InferenceX change. Consider adding a git checkout <sha> right after the clone to pin reproducibility.
Extended reasoning...
The bug: The new elif branch added for FRAMEWORK==dynamo-sglang && MODEL_PREFIX==dsv4 && PRECISION==fp4 (runners/launch_b300-nv.sh:82-84) does:
git clone --branch main --single-branch https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
cd "$SRT_REPO_DIR" || exit 1with no follow-up git checkout <sha> to pin a revision. From that point forward, whatever srtctl apply runs, and whatever recipes/dsv4-pro/sglang/b300-fp4/8k1k-mtp.yaml matrix selectors resolve to (override_tp4_tp4, zip_override_dep4_dep8[0..5], etc., as referenced by the new dsv4-fp4-b300-dynamo-sglang-mtp config entries in configs/nvidia-master.yaml), is whatever happens to be on NVIDIA/srt-slurm's main branch at the moment CI runs — not a fixed, known-good state.\n\nWhy this isn't obviously fine: One verifier raised a fair refutation — several other branches in this same if/elif chain also track floating branches rather than pinned commits: the dynamo-vllm dsv4 case checks out aflowers/vllm-gb200-v0.20.0, and the generic minimaxm3 / final-else cases check out sa-submission-q2-2026. So it's not literally true that every branch pins an exact commit SHA, and tracking a named branch is an established pattern here, not unprecedented.\n\nHowever, the most directly comparable case undercuts that defense: the minimaxm3-mtp branch (added earlier, also on B300, also git clone --branch main --single-branch) does the exact same clone-from-main, and then immediately adds git checkout c1b6b5c97f323baefad577d70c4e8392b6f537d9 to pin it. That's the one other place in this file that clones main specifically (as opposed to a purpose-built long-lived feature/submission branch like sa-submission-q2-2026 or aflowers/vllm-gb200-v0.20.0), and it treats main as too volatile to leave unpinned. This new dsv4 dynamo-sglang branch clones the same main and skips that step, which is the actual inconsistency — not 'every branch pins a SHA' but 'every branch that clones the fast-moving default branch also pins a SHA afterward, except this one.'\n\nConcrete failure mode: Suppose next month NVIDIA/srt-slurm's maintainers rename or restructure recipes/dsv4-pro/sglang/b300-fp4/8k1k-mtp.yaml (or change what override_dep4_tp4/zip_override_dep4_dep8[N] select) as part of unrelated matrix cleanup. The next CI run of this InferenceX config silently picks up that change: CONFIG_PATH=${CONFIG_FILE%%:*} would either fail to resolve (caught by the new existence check added in this same PR) or, worse, resolve to a matrix entry whose topology/settings have shifted, producing a benchmark result that doesn't match what was recorded when this PR was authored — with zero corresponding diff in this repo to explain the change. That defeats the reproducibility goal that the pinned-SHA convention elsewhere in this file exists to protect.\n\nFix: Add a git checkout <commit-sha> immediately after the clone/cd, mirroring the minimaxm3-mtp pattern, pinned to whatever commit on NVIDIA/srt-slurm currently contains PR #281's merged matrix.\n\nSeverity: This is a nit, not a blocker — the run works correctly today (assuming srt-slurm#281 is already merged to main), and the failure mode is future drift rather than an immediate correctness problem at merge time.
中文：将 B300 DSV4 Dynamo-SGLang 配置指向服务器上预置的 DeepSeek-V4-Pro-NVFP4 模型路径。
固定 srt-slurm 配方版本，确保 DSV4 B300 Dynamo-SGLang 运行使用可复现的配置。
修复（dsv4）：固定包含 UCX CUDA 传输配置的 srt-slurm 版本。
中文：将当前 main 合并到 DSV4 SGLang MTP 配置分支。
Use the stock DeepSeek-V4-Pro FP4 checkpoint, correct recipe labeling, and pin the srt-slurm checkout.\n\n中文：使用标准 DeepSeek-V4-Pro FP4 检查点，修正配置标注，并固定 srt-slurm 版本。
RohitNagraj
changed the title
Pin NVIDIA/srt-slurm main, overlay the checked-in recipes, and enable CUDA-aware UCX transport for every MTP topology. 修复：固定 NVIDIA/srt-slurm main 提交，覆盖仓库内置配方，并为所有 MTP 拓扑启用支持 CUDA 的 UCX 传输。
同步最新 main 分支，并将 DSV4 MTP 变更日志条目保留在文件末尾。
RohitNagraj
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30431632620 |
|
/reuse-sweep-run 30431632620 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
|
/reuse-sweep-run |
# Conflicts: # perf-changelog.yaml
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
…gm 能耗采集到 gb dsv4 与 qwen3.5 fp4 (#2507) * [Power] feat: extend dcgm energy lanes to gb dsv4/qwen3.5 fp4 / 扩展 dcgm 能耗采集到 gb dsv4 与 qwen3.5 fp4 recipe 侧:5 个 DSV4 8k1k 点位与 qwen3.5-fp4 gb300 lowlat 配方加 telemetry 块(gb300 用 19401 端口)。launcher 侧:gb200 dsv4 分支在 power 时改克隆 pinned producer;gb300 新增仅 power 生效的 dsv4 路由(非 power 路径保持原样);precision 门换成 framework 门(dynamo-sglang 之外仍然快速失败)。契约测试同步:recipe 白名单 8 项 + 逐配方 telemetry 断言。 * [Power] fix: no-preflight + numa1 model path for node-local fp4 power lanes / 为 node-local fp4 能耗 lane 加 no-preflight 与 numa1 模型路径 两个 canary 都死在同一处:模型在 compute-node 本地盘(gb200 /mnt/numa1、gb300 /scratch),login 侧 runner stat 不到,srtctl preflight 必然误报。沿用 repo 已有豁免机制(agentic/glm5.1/qwen3.5-fp8 同款),范围限定 USES_DCGM_POWER=1;gb200 power lane 的 MODEL_PATH 指向 agentic 与 llm-d 实测在用的 /mnt/numa1 检查点,共享的 lustre 路径保持不动。 * [Power] fix: bump gb300 dsv4 power recipes to a live sglang nightly / gb300 dsv4 能耗配方镜像换成仍在线的 nightly 原 pin nightly-dev-cu13-20260520 已从 Docker Hub 过期删除(404),gb300 无 sqsh 缓存,lane 无法拉起。换成 B300 DSV4 家族(#2362/#2363)在用且仍在线的 nightly-dev-cu13-20260721-8905cbd4;仅动两个 power 点位,其余 gb300 dsv4 配方保持原 pin。 * [Power] fix: bump dead gb300 dsv4 image pin in nvidia-master / 修正 nvidia-master 中已失效的 gb300 dsv4 镜像 pin launcher 的 import_squash 用的是 master yaml 的 image 字段,0520 nightly 已被 Docker Hub GC;与配方一致换成仍在线的 0721(B300 DSV4 家族同款)。仅动 dsv4-fp4-gb300-dynamo-sglang 一个 key,mtp key 不碰。 * fix(power): sync dsv4 gb300 images and drop dead launcher branch * fix(changelog): restore entry separator lost in merge
4 participants
Summary
deepseek-ai/DeepSeek-V4-Procheckpoint staged at/scratch/models/DeepSeek-V4-ProNVIDIA/srt-slurmfrommain, pin commitc180328b98c3793ca84a1e24a030f90545eb7d5d, and overlay eight recipes checked into this repositoryuse_chat_template: trueandUCX_TLS=cuda_copy,rcfor every topology, without aUCX_NET_DEVICESallowlist--no-preflightscoped to the compute-node-local/scratch/modelspathmainafter it is merged中文说明
deepseek-ai/DeepSeek-V4-Pro检查点，节点本地路径为/scratch/models/DeepSeek-V4-Promain克隆NVIDIA/srt-slurm，固定到提交c180328b98c3793ca84a1e24a030f90545eb7d5d，并覆盖本仓库内置的八个 recipeuse_chat_template: true和UCX_TLS=cuda_copy,rc，且不添加UCX_NET_DEVICES白名单/scratch/models路径启用--no-preflightmain中的 recipe